Concept Formation From Very Large Training Sets

This paper proposes an a l t e r n a t i v e to Quin lan 's a lgor i thm fo r forming c l a s s i f i c a t i o n t rees from large sets of examples. My a lgor i thm is guaranteed to te rminate . Quin lan 's a lgor i thm i s usua l ly f a s t e r . I. The Nature of the Problem. We have a populat ion of objects which we want to c l a s s i f y i n to two** groups. We have a set of a t t r i b u t e s , each w i th a sna i l f i n i t e number of d i s t i n c t va lues , and a set of examples whose a t t r i b u t e s have been measured and which have al ready been c l a s s i f i e d . Our goal is to f i nd a r u l e , based on these examples, which we can use to c l a s s i f y other members of the popu la t ion . In general t h i s w i l l lead us to s t a t i s t i c a l methods such as c l us te r analys is ( E v e r i t t , 1974, Kenda l l , 1975, S t u r t , 1981a, S t u r t , 1981b). The l a rge r our c o l l e c t i o n of examples, the more l i k e l y i t is that some of them are m i s c l a s s i f i e d . ( S t u r t , 1981a) provides an exce l len t i l l u s t r a t i o n of how improving the f i t of a ru le to the t r a i n i n g set (beyond a ce r t a i n po in t ) can make It perform worse on the rest of the popu la t ion . Even so, there are i n t e r e s t i n g tasks where the domain is f o rma l , and we can be sure that we have a l l the in fo rmat ion we need and that our c l a s s i f i c a t i o n s are c o r r e c t . There a re , however, i n t e r e s t i n g problems where the domain is formal ra the r than r e a l , and we can be sure that a l l the re levant in fo rmat ion is ava i lab le and our c l a s s i f i c a t i o n s are c o r r e c t . Chess pos i t ions and a lgebra ic equations are two such domains. *Th is work was supported by a Commonwealth Scho larsh ip . Computing was done on the SERC Dec-10 under grant CR/C/20826 ** ID3 and ray a lgor i thm are both presented in terms of two ca tego r ies . The in fo rmat ion h e u r i s t i c genera l ises to any f i xed number of ca tegor ies , and so do the a lgo r i thms.